Least Angle Regression and LASSO for Large Datasets

Least-Angle Regression and the LASSO (`1-penalized regression) offer a number of advantages in variable selection applications over procedures such as stepwise or ridge regression, including prediction accuracy, stability and interpretability. We discuss formulations of these algorithms that extend to datasets in which the number of observations could be so large that it would not be possible to access the matrix of predictors as a unit in computations. Our methods require a single pass through the data for orthogonal transformation, effectively reducing the dimension of the computations required to obtain the regression coefficients and residual sums-of-squares to the number of predictors, rather than the number of observations.